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Abstract. Finding a sparse representation of a possibly noisy signal can be modeled as 
a variational minimization with £q-sparsity constraints for q less than one. Especially 
for real-time, on-line, or iterative applications, in which problems of this type have to be 
solved multiple times, one needs fast algorithms to compute these minimizers. 

Identifying the exact minimizers is computationally expensive. We consider minimiza- 
tion up to a constant factor to circumvent this limitation. We verify that g-dependent mod- 
ifications of shrinkage rules provide closed formulas for such minimizers. Therefore, their 
computation is extremely fast. We also introduce a new shrinkage rule which is adapted 
to q. 

To support the theoretical results, the proposed method is applied to Landweber iter- 
ation with shrinkage used at each iteration step. This approach is utilized to solve the 
ill-posed problem of analytic ultracentrifugation, a method to determine the size distribu- 
tion of macromolecules. For relatively pure solutes, our proposed scheme leads to sparser 
solutions with sharper peaks, higher resolution, and smaller residuals than standard regu- 
larization for this problem. 

Keywords, shrinkage, variational optimization, sparsity, frames, Fredholm integral equa- 
tions. 
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1 Introduction 

Decomposing signals into simple building blocks and reconstructing from shrinked 
coefficients are used in signal representation and processing, e.g., wavelet shrink- 
age is applied to noise and clutter reduction in speckled SAR images, cf. [38]. 
Statistical approaches and Bayesian objectives for noise removal make use of var- 
ious siirinkage strategies, cf. [18,24,26,46]. Variational models as in [9] justify 
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shrinkage by smoothness estimates of the unperturbed signal. Other shrinkage 
rules are derived from a diffusion approach in [37]. 

Signal approximation with sparsity constraints leads to variational minimization 
problems, and the denoising approach in [9] is a particular case. The expression 
to be minimized is a sum of an approximation error and a penalty term which 
involves weighted ^^-constraints, see Section 2.1. In [14], iteratively shrinking 
coefficients of an orthonormal basis expansion provides a sequence converging to- 
wards the minimizer. The method covers the convex case q e [1,2], but sparse 
signal representation, coding, signal analysis, and the treatment of operator equa- 
tions require the consideration of redundant basis-like systems and the nonconvex 
case q e [0, 1) as well, see for instance [2, 10, 12, 13, 23, 28, 42] and references 
therein. By using hard-shrinkage, the algorithm in [14] converges towards a local 
minimum for q = cf. [4]. Under the restricted isometry property (RIP) [8,17], 
the iteration converges towards the exact minimum, cf. [3]. However, RIP does 
not hold in many situations and therefore the local minimum could be far off the 
global minimum. The approach does not cover q e (0, 1), and, for applications 
where computation time is crucial, a faster algorithm is desirable. 

In the present paper, we obtain complementary results for g e [0, 1 ) in terms of 
minimization up to a constant factor. In fact, we verify that such a minimization 
can be derived from (/-dependent modifications of shrinkage rules. This means we 
have a closed formula for these minimizers, which allows for a fast computation. 
We also introduce new shrinkage rules which are adapted to q. We then propose 
a Landweber iteration with these new shrinkage strategies applied in each step to 
treat sparsity constraints for q G (0, 1), cf. [4, 14] for soft- and hard-shrinkage. 
This approach is then applied to the ill-posed problem of sedimentation velocity 
analytical ultracentrifugation, a method to determine the size distribution of inter- 
acting macromolecules [11,41]. Its physical model leads to a Fredholm integral 
equation that needs to be regularized. For highly pure monomers, the solution is 
expected to be highly sparse with few sharp peaks. Our numerical experiments 
suggest that our proposed iterative shrinkage scheme leads to sharper peaks, fewer 
nonzero entries, and smaller residuals. Thus, it provides a useful add-on to stan- 
dard analytical ultracentrifugation analysis, cf. [7,41]. 

The outline is as follows: In Section 2, we present the variational problems un- 
der consideration and we recall the concept of frames. We introduce shrinkage 
rules in Section 3. The main results about minimization up to a constant factor 
are presented in Section 4, and in Section 5 we apply the results to sparse signal 
representation. We introduce a new family of shrinkage rules in Section 6. The 
modified Landweber iteration is explicitly introduced in Section 7, where we also 
present numerical results about sedimentation velocity analytical ultracentrifuga- 
tion. Conclusions are given in Section 8. 
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2 Variational Problems and Frames 

2.1 Variational Minimization Problems 

Let 1/ be a bounded operator between two Hilbert spaces "H and l-L' , and let 
{/n}neA^ be a countable collection in l-L. Given h e l-L' , we consider the min- 
imization problem 

^^{\\h-Lg\W,+ Y,an\{gJn)V). (2.1) 

where q G (0,2], (an)n6A^ is a sequence of nonnegative numbers, and (■, ■) de- 
notes the inner product on H. This makes also sense for 5 = with the convention 
0" = 0, and the penalty term then counts the nonzero entries of {{g, fn))neAf 
weighted by {an)neJV- For H = H' and L = id^, problem (2.1) is relevant in 
wavelet based signal denoising. There, {fn}neM is a wavelet system, and the spar- 
sity constraint on the right hand side of (2.1) is related to the Besov regularity of 
the signal to be recovered, see [9] for details. Our approach is neither restricted 
to L being the identity nor must L be injective. However, we assume that it has a 
bounded pseudo inverse, i.e. there is a bounded operator L* : H' ^ H such that 
LL^L = L. Thus, we first consider well-posed problems and are therefore more 
restrictive than in [14]. Nevertheless, we address ill-posed problems in Section 7 
by extending the iterative shrinkage procedure introduced in [14]. 

The sequence {an)neJV is a collection of variable parameters which must be 
fitted to h and L. If all components of q;„ = o; are identical, then 

a^{\\h-Lgm^,,J2\{9''J~n)n (2.2) 

n€Af 

is considered as a curve in M?, where g" is a minimizer of (2.1), and one finally 
chooses a according to a point of maximal curvature, see [33] and [36] for the 
L-curve and H-curve criterion, respectively. It requires to compute minimizers 
(/" for many different values of a, and g°' must be efficiently computable. This 
is another motivation for avoiding costly iterative minimization schemes beside 
real-time and on-line applications. 

Handling nonstationary noise requires that {oin)neM depends on n, but it is 
often still reasonable to assume that there are positive constants a and h such that 

a<an<h, for all n e A/". (2.3) 

2.2 Bi-frames 

The singular value decomposition of L is considered in [35] to address q S [0, 1). 
The system {/„} in (2.1) is supposed to be an orthonormal basis for H which 
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diagonalizes L. However, diagonalizing L can be extremely difficult in practical 
applications. We will consider redundant basis-like systems, and L is not required 
to be diagonalized: a countable collection {fn}neAf in ^ is a frame if there are 
two positive constants A, B such that 

A\\9\\n< Y.\^9Jn)\^ <B\\g\\\,iox2A\gen. (2.4) 

If {/njnGAT is a frame, then its synthesis operator 

F : i2{M) ^ {cn)neM ^ 5^ c„/„, (2.5) 

is onto. Each g ^ T-L then has a series expansion, but we still have to find its 
coefficients. The synthesis operator's adjoint 

F*:n^l2{M). 9^{{9jn))neAf (2.6) 

is called analysis operator, S = FF* is invertible, and {S~^ fnjneAf is called 
canonical dual frame and expands 

g= J2i9,S-'fn)fn, for M gen. 

The inversion of 5" can be difficult, and, since F need not be injective, there could 
be 'better' coefficients than {g, fn)- This motivates the following: two frames 
{fn}n€Af {fn}neAf ^rc Called a pair of dual frames (or a biframe) if 

g=Y.{gJn)fn,iox2A\geH, (2.7) 

i.e., FF* = id>^, where F* is the dual frame's analysis operator. For instance, the 
canonical dual of a wavelet frame may not have the wavelet structure as well, but it 
can possibly replaced by an alternative dual wavelet frame, cf. [ 16, 19, 21 ,22, 3 1 ,40] 
and references therein. 

Throughout the paper, we suppose that {fn}n€M and {/njneA/" are a bi-frame 
forU. 

3 Shrinkage Rules 

To solve (2.1), shrinkage plays a crucial role. Following ideas in [43], we call a 
function g : C x M>o ^> C a shrinkage rule if there are constants Ci,C2, p, D > 
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such that both conditions 

\x — g{x, a)| < Ci min(|x|, a), for all a > 0, x e C, (3.1) 
\gix,a)\ < C2\x\\-\^, for all a > 0, |x| < Da, (3.2) 



are satisfied. While (3.1) forces g{x,a) to be close to x for small a, condition 
(3.2) means that q{x, a) has sufficient decay as x goes to zero. A shrinkage rule 
Q is called a thresholding rule if there is a constant C3 > such that |x| < C^a 
implies Q{x,a) = 0. A thresholding rule allows for p = 00 in (3.2), where we 
use a°° = OifO<a< 1. We will recall a few common shrinkage rules and we 
restrict us to a; e M, see also Figure 1: Soft-shrinkage is given by Qs{x, a) = {x — 

sign(a;)a)l^l^l>Q,j.. Contrary to soft- and hard- shrinkage Qh[x,a) = 

2 

the nonnegative garotte-shrinkage rule Qg{x, a) = {x— ^)l{|a-|>Q} is continuous 
and large coefficients are not changed much. It has been successfully applied to 
image denoising in [29]. Similar properties has hyperbolic-shrinkage Q^y{x, a) = 
sign(x) - a2l{|^|>„|(x), cf. [43]. 

2n+l 

The n-degree garotte-shrinkage rule is given by ^"(a;, a) = ^Jq:^. see [43]. 
For /c e N, the twice differentiable rule 

\x\ < a 



^,,(x,a)= „ ' , (3-3) 

- sign(a;)(a - 2fc+x), \x\ > a 

is considered in [46]. Both rules are shrinkage rules with p = 2k = In and 



C2 = 1. The rules Q{x,a) = x(l - ■y/^ifjj^) and Q{x,a) w xexp(-0.2^) are 
based on diffusion, see [37]. One verifies that both are shrinkage rules with p = 1, 
and we refer to them as diffusion 1 and 2 in Figure 1. 
Bruce and Gao proposed firm-shrinkage 

Qf{x,ai,a2) = xl{u|>Q,} + sign(a;) 1{q,<|x|<q,} 

— Oil 

in [30]. For fixed ai, the mapping {x, a) 1— > Qf{x, ai,a) is a thresholding rule. 



4 Main Results 

For q e [0,2], let £^^"'\M) denote the weighted £g(A^)-space, i.e., the space of 
complex-valued sequences {ujn)n€M such that := S„g^Q!„|w„|^ is fi- 

nite. One observes that a„| (5, /n) |^ = ||^*5||%„): and to shorten notation, 

we denote 



Jq{h,g) = \\h-Lg\\l^, + \\F*g 
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(a) hard (b) soft (c) nonnegative (d) hyperbolic 

garotte 




(e) 1 -degree garotte (f) 2-degree garotte (g) k = 1-shrinkage (h) k = 2-shrinkage 




(i) diffusion 1 (j) diffusion 2 (k) firm, oi = 1 (I) firm, ai = 2 



Figure 1. Shrinkage rules g{x, a), for q = 3. (a) is not continuous. (b)-(d),(k), and 
(1) are continuous but not differentiable. (e)-(j) are smooth 

The idea for the following main result is to replace a shrinkage rule g{x, a) by its 
(^-dependent expression g{x, a|x|'~'). Due to (3.1), it vanishes as a; 7^ goes to 
0, and we apply g{x, a\x\'^~^) = for x = 0. If p = 00, we use ^ = 0. Since 
gg{x, a) = gs{x, a^\x\~^), the nonnegative garotte is (/-dependent soft-shrinkage 
for g = and a replaced by a^. It turns out that (/-dependent shrinkage expressions 
provide minimizers of (2.1) up to a constant factor: 

Theorem 4.1. Let g be a shrinkage rule with p G [j, oo\. Suppose that F*L*LF 
is bounded on i^^^'^\j\f). Let q = ^, then there is a constant C > such that for 
all h e range (I/), and for all g eV. 

Jq{h,g)<CJg{h,g), 

where g = L^LF g{vn, Oin\vn\'^~^)neN' with v = F*L*h. 

IfF*Fis also bounded on ^["^''(A/"), one can choose g = F g{vn, an\vn\'^~^)n€N'- 
If (2.3) holds, then the statements extend to all q G [^,2], and C is independent of 
1- 
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Remark 4.2. If the bi-frame is biorthogonal and F£^^^'^^ c range(L*I/), then 
F*L*LF = id (o„), because F*F is the identity and L*L is the identity on its 
range. The boundedness condition is then trivially satisfied as it is for finite J\f. 

To prove Theorem 4.1, we consider a decoupled minimization problem: given 
V G (-li-M), we try to minimize 

Xq{v,uj) = {\\v - Lo\\\ + ^ a„|w„|«) (4.1) 

over a; e hi-^)- It turns out that minimizing (2.1) and (4.1) up to a constant factor 
are equivalent: 

Proposition 4.3. Given q e [0, 2], suppose that F*V*LF is bounded on 4""' (A/"). 
For h G range(L), let v = F*L*h. If Cj minimizes (4.1) up to a constant factor, 
then g = L^LFuj minimizes (2.1) up to a constant factor. 

If F*F is bounded on il^"'\j\f), one may also choose g = Flu. The reverse 
implication holds for u) = F*V*Lg and Co = F*g, respectively. 

Given a parameter set F and two expressions (ay)ygr and (67-)xgr such that 
there is a constant C > with Ur < Cbr for all r £ F, we write Ut ^ W in the 
following proof. 

Proof of Proposition 4.3. Let tli minimize (4.1) up to a constant factor, i.e.,Iq{v, ui) < 
Iq{v,uj), for all u! E iiiJ^)- Since FF* = idy, and since LL*L = L yields 
LL^h = h, we have h = LFv. Applying LL*L = L implies Lg = LFuj, which 
leads to 

Jqihrg) = \\LFv - LFibWl,, + \\F*L'LFu^r 

Since LF : £2 ^ W is bounded and due to the boundedness of F*L*LF on 
this implies J'q{h, g) < Tq{v, Cj). Since Co minimizes (4.1) up to a constant factor, 
we have Jq{h,g) < Iq{v,F*L^Lg), for all g e H. By applying that F*L* is 
bounded and that FF* = id-y, we obtain, for all g ^l-L, 



Jq(hrg) < \\F*L*h - F*L*Lg\\l + \\F*L'LFF*gr 



< \\h - Lg\\l + \\F*L*LFF*gr < Jq{h,g), 

where we have used that F*L*LF is bounded on 

Analogous arguments can be applied to the case g = FCj, and the reverse im- 
plications follow in a similar way. □ 
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Next, we obtain a solution of the discrete problem (4.1). 

Proposition 4.4. Let g be a shrinkage rule with p G oo\. Then there is a con- 
stant C > such that for all g G 2], for all v E ^2(A/'), and for all w E ^2 (A/"), 

Iqiv,Oj) < CIq{v,Uj), 

where ui = a„|w„|«~')„gA/-. 

Remarlt4.5. Theexactminimizerof (4.1)for5 = 2is known to be [ ^^^ "^^neM' 
However, (a;, a) ^ is not a shrinkage rule since (3.2) is violated. On the 

other hand, the rule q{x, a) = j^^x is a shrinkage rule with constant p = I. The 

^-dependent expression q{x, a|x|'~') for q = 2 then yields the exact minimizer. 
In this sense the exact minimizer for g = 2 is still derived from shrinkage. 

Proof of Proposition AA. First, we consider ^ < p < 00. Due to (3.1), the se- 
quence Q{vn, oin\vn\'^~^)neM IS indeed contained in £2(A/'). Adapting results in [9] 
to our setting yields that the hard-shrinked sequence Qh{vn, Oin\vn\'^~^)neAf mini- 
mizes (4.1) up to a constant factor. By using the short-hand notation 

Kn ■■= \vn - gh{Vn,an\Vn\'^~^)\^ + | | ^n^"' ) | 

Gn ■■= \Vn - g{Vn,an\Vn\'^~^)f + | a„ |w„ j"^"' ) |'', 

we consider each n in the sequence norms separately. We aim to verify G„ < Kn 
independently of n. For f„ = 0, we have G„ = Kn- Now, we suppose w„ 7^ 0. 
Since (3.2) gets weaker as p and D decrease, we may assume that q = ^ and 
D <l. Case 1: For |i;„| < Dan\vn\''~\ (3.1) and (3.2) with p = ^ yield 



an\v„\i-^ 



Case 2: For > Dan\vn\'^~^ , we have l/D > a„|i>„|^~^, and the estimate 
(3.1) yields 

G„ < + + Ci min(|v„|, a„|f„|«-'))'^ 

< C^an\vn\''an\vnr^ + a„|«„|«(l + Ci a„ |«„ 19-2)9 

< C^vnl'^jj + (1 + Ci/DYaM'^ < aM'^ < K„/D. 

Hence, Gn ^ Kn holds in both cases. Similar arguments verify the statement for 

p = oo. n 
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Our main result follows from combining both propositions: 

Proof of Theorem 4.1. According to Proposition 4.4, g{vn,an\vn\'' ) 
minimizer of (4.1) up to a constant factor, where v = F*V* f. For q= Proposi- 
tion 4.3 then implies Theorem 4.1. If (2.3) holds, F*L*LF and F*F are bounded 
on Interpolation between ^J"^'' and £2""' yields uniform boundedness on 

4""),for<?e[i,2]. □ 

Remark 4.6. We did not use the Hilbert space structure of l-L' and in fact Theorem 
4.1 still holds if l-L' is a (quasi) Banach space. 

5 Sparse Approximation 

Given h (possibly noisy) and a frame {fn}neM for important problem 
in sparse signal representation is to find the minimizer of 

min llwlU subject to Fcj w /i, (5.1) 

for q S [0, 1). Under additional requirements on {/nlneA/" ^^id h, the solution for 
g e [0, 1) can be obtained from solving the much simpler convex problem with 
q = 1, cf. [8, 17]. However, these results are limited to finite J\f, and the additional 
requirements are not satisfied in many situations. 

The problem (5.1) is often replaced by a variational formulation, and one seeks 
to minimize 

ICq{h,Oj) = \\h - Fu}\\l^ + ^ an\ujn\'' 

over uj S £2 (A/"). For finite A/", £g -basis-pursuit as in [44], for instance, solves (5.1) 
by minimizing over all pseudo inverses F*. The associated variational 

formulation is 

min ( min {\\F*h — uj\\e,^_ + q;„|w„|'^)) . 

We do not require J\f to be finite, and instead of minimizing over F*, we suppose 
to have a particular pseudo inverse F* being the analysis operator of a dual frame 
{/„}„gAr such that F*F is bounded on 4""' (A/"): 
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Figure 2. The curve Cg, for q e [0, 1], is continuous and continuation yields Cq = 1 
and Ci = ^ . 

Theorem 5.1. Given a bi-frame {fnjneAf '^^d {fn\n€N'' g be a shrinkage rule 
with p G [^jCxd]. Suppose that F*F is bounded on ^J"^'(A/'). Let q = ^, then 
there is a constant C > such that for all h E H and for all lj G ^2(A/') 

ICq{h,Lj) < CICq{h,U!), 

whereCb = F*Fg{vn,an\v„\'^~^)n€J^withv = F*horui = g{vn,an\vn\'^~^)neAf- 
If (2.3) holds, then the statement extends to all q E [^,2], and C is independent of 

q- 

Remark 5.2. For sufficiently smooth wavelet bi-frames with vanishing moments, 
the operator F*F is bounded on provided that (a„)n6A/' satisfies (2.3), cf. [20]. 

Proof. We replace V., T-i' , L, L*, and the bi-frame {fn}neAf' {fnjneAf in (2-1) by 
£2(A^), "H, F, F*, and the canonical basis {enjneAf for ^2(A/'), respectively. The 
condition on F*L*LF in Theorem 4.1 becomes is bounded on ^["^'(A/")', 

and Theorem 4. 1 implies Theorem 5.1. □ 

6 Explicit Shrinkage Rules Between Hard- and Soft-Shrinkage 

This section is dedicated to finding a family of shrinkage rules which is adapted 
to q in (4.1). For q E [0, 1), we will use Cq = 2'?"^ ■ It is monotonically 

decreasing with co = 1, and continuous extension yields ci = ^, see Figure 2. 

Due to [1], the exact minimizer of (4.1) is sandwiched between soft- and hard- 
shrinkage. Let us introduce the new shrinkage rule 

Bhli^^") = (x - sign(x)gcga)l||^[>„c,}, (6-1) 

see Figure 3. 
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(a)g = 0.1 (b) 9 = 0.3 (c)q=l/2 (d) g = 0.8 



Figure 3. Shrinkage rule g^l.{x, a) for a = 4. It tends to hard-shrinkage for q\0. 
Soft-shrinkage is approximated by g 1. 




(a)g = 0.1 (b) 9 = 0.3 (c) g = 1/2 (d) g = 0.8 



Figure 4. g-dependent expression qI^\{x, a|a;|' ') for a = 4. According to (6.2), 
the smaller q the faster tends this expression to x. 

One easily verifies that Q^^\{x, a) has a jump of size (1 — q)cqa and, for |x| > 
Cqa, we have 

\x - Qylil{x,a)\ = qcqa. 

On the other hand, the ^-dependent expression Q"^\{x, Q!|a;|'~' ), see Figure 4, has 

1 1 
a jump of size (1 — q){cqa)--i and, for |x| > {cqa)^-i, we obtain 

|x - ^^(x, a\x\i-^)\= qCqa\x\i-\ (6.2) 

Hence, for e [0, 1), the difference goes to zero as x goes to infinity. 

The following theorem says that the new rule (6.1) is well adapted to 5 e [0, 1]: 

Theorem 6.1. The sequence Q\^\{vn, Oin\vn\''~^)neAf '■^ exact minimize r of (4.1) 

at the endpoints q = 0, q = 1. It minimizes (4.1) up to a constant factor in be- 

1 1 

tween, and it coincides with the exact minimizer on {n M : \vn\ < Cq~'' }■ 

Proof. Soft-shrinkage Qs{vn, ^)n€JV is the exact minimizer of (4.1), for q = 1, 
cf. [9]. Note that 

0s{Vn, = Qhli'^n, Ctn), for all 71 eM. 
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The exact minimizer for g = is hard-shrinkage gh{vn, y/fy^jneM, see [35], and 
we have the identity 

Qh{vn,V(^) = QT,si'^''i,an\vn\~^) for all n e A/". 

The shrinkage rule g'^^^ satisfies (3.2) for p = oo. Hence due to Proposition 4.4, it 
minimizes (4.1) up to a constant factor. 

Wehave£)|^'^(u„,a„|i;„|9-i) =Oiff\vn\ < Cqan\vn\'^~^ ■ Since |f„| < Cqan\vn\ 

is equivalent to l^np"' < Cga„, it is also equivalent to |f„| < Cq~'' an~'' , for 

q G (0, 1). According to the results in [35], see also [1], each exact minimizer 

1 1 

{u}n)n€Af satisfies ujn = for \Vn\ < cf an'' ■ u 

Due to Theorem 6.1, the rule is an adaptation to q e [0, 1]. This might also 
be useful for parameter fitting: While a = {an)neM can be fitted to / and L by 
considering (2.2), the new family g^lf'^ provides additional flexibility to optimize 
the choice of q as well. One optimizes a = a{q) as in (2.2), one may then vary 
q S [0, 1] and may optimize this sparsity parameter by analyzing the univariate 
curve a{q). 

7 Iterative Shrinkage Strategies 

In the present section, the derived shrinkage strategies in Section 6 are applied to 
inverse problems. We slightly change our perspective and consider the problem 

argnun(||/-rff||) (7.1) 

in which the operator T does not have a bounded pseudo inverse or the norm is 
extremely big. Such an ill-posed problem needs regularization. During the last 
decade, regularization with sparsity constraints has attracted significant attention, 
see, for instance, [5, 6, 14, 15, 25, 27, 35, 39]. One solves 

argnun(||/-r5||' + a</.(ff)), (7.2) 

where (/) is a measure of the sparsity of g in some chosen dictionary, and the non- 
negative regularization parameter a weights the sparsity term. 

7.1 Landweber Iteration with Shrinkage 

A shrinked Landweber iteration has been developed in [14] to minimize (7.2) for 
the term (j){g) = \gk\^, provided that 1 < (? < 2 and {gk)k are the coefficients 
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for g's representation in an orthonormal basis. To reduce notation, let us assume 
that / and g are already discretized and hence are just sequences. For q = 1, the 
term (j){g) enforces sparsity. The minimization (7.2) then is well-posed, and the 
iteration given by 



g^+' = Sa{g' + T*f - T*Tg^), where 5„(x)„ = a), (7.4) 



converges towards the minimizer of (7.2), see [14]. Soft-shrinkage occurs in this 
iterative scheme, because it is the exact minimizer of (4.1). To address other 1 < 
q < 2 in (7.2) with (f){g) = J2k ISfcl'^' need to apply the shrinkage rule that 
corresponds to the exact minimizer of (4. 1) for this particular q. On the other hand, 
it is shown in [45] that (7.2) with the stronger sparsity requirements < p < I 
is still well-posed and a regularization of the original problem (7.1). However, 
the nonconvexity of (j){g) in this case makes it difficult to design a numerically 
attractive algorithm for the actual minimization. The g-dependent expression of 
the shrinkage rule g'^'^^ is not the exact minimizer of (4.1), but still a minimizer up 
to a constant factor. Motivated by the results in Section 6, we propose to replace 
soft-shrinkage with the (j'-dependent expression of q^'^\ i.e., to replace Sa in (7.4) 
with 

This modified scheme is known to converge for 5 = and q = 1, cf. [4, 14]. It is 
thus reasonable to believe that it also converges for < g < 1, which is supported 
by numerical experiments. 

7.2 Analytic Ultracentrifugation 

Sedimentation velocity analytical ultracentrifugation is a method to determine the 
size distribution of macromolecules in a solute, cf. [7, 41]. The physical model 
leads to a Fredholm integral equation 



whose kernel K of the integral operator T represents the sedimentation profile and 
is only implicitly given through the solution of the Lamm equation, a differential 
equation discussed in [34]. From the experimentally observed signal /, one must 
deduce the particles' or macromolecules' size distribution g. However, this is an 
ill-posed problem and requires regularization. State of the art regularizations for 
this problem are Tikhonov and maximum entropy regularization in [11, 32,41]. 
Both methods have also been used in combination with Bayesian priors [7]. 




(7.3) 




(7.5) 
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Partial information about a solute is often available. We consider the case in 
which we know a-priori that the solute is well separated into molecules of very 
different sizes. In other words, the seeked size distribution g is sparse, i.e., has 
only few peaks and is almost zero elsewhere. The sparser the expected distribution 
the smaller we may want to chose q. However, there is a trade off, because we then 
only minimize up to a constant factor. It seems reasonable to believe that heuristics 
can be developed to chose a near optimal q for a given experiment. 

Before we apply the proposed iterative scheme to solve the analytical ultracen- 
trifugation problem, we observe that the size distribution g must be nonnegative. 
We first discretize (7.5) by sampling on a finite grid. By using the nonnegativity 
as an additional regularization, we modify the application of the shrinkage process 
Sa in such a way that negative arguments are not shrinked in its original sense, but 
simply set to zero. It does not introduce any additional discontinuities, because 
q{x, a) ^> as < a; ^> 0. This procedure enforces a nonnegative limit. 

In our numerical experiments, we consider different values of < g < 1 and 
compare the results in terms of how much sparsity we obtain while only introduc- 
ing a relatively small residual ||/ — Tg\\- We finally compare these findings to 
maximum entropy regularization that was used in [7,41] to solve the ill-posed 
problem of analytical ultracentrifugation. We aim to verify that our proposed 
scheme can provide sparser solutions with sharper peaks, higher resolution and 
smaller residual. 

7.3 Numerical Results 

Maximum entropy regularization 

argnun - T g\\l + ^Y^gn H9n)) 

n 

is the state of the art tool to solve (7.5) for the analytical ultracentrifugation, cf. [7, 

41]. It has been implemented in the softwaretool SEDFIT [41], that we use as a 

reference. SEDFIT uses f-statistics to choose /?. 

Sharper spikes, fewer nonzero entries, and higher resolution: 

Our scheme is applied to a highly pure IgG antibody solute. Due to the purity, 

the "correct" solution to the underlying Fredholm integral equation must be highly 

sparse. We use 100 measurements on an equidistant grid to solve the discrete 

analogue of the integral equation (7.5). The residual ||/ — TgsWtj the SEDFIT 

solution gs is 0.7878. Although the solution seems sparse, cf. the red graph in 

Figure 5, it has many small entries and the spikes are relatively wide. 

The choices < <? < 1 promote sparsity and, for sufficiently small a > 0, 
our proposed scheme leads to smaller residuals, sharper spikes, and few small 
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Figure 5. Comparison between SEDFIT and our proposed scheme with q = 0.3: 
we obtain sharper peaks, especially around 16 and 23. While SEDFIT is nonzero 
between 42 and 90, the solution to our scheme is zero except for peaks at 45, 55, 66, 
and 89. 



entries, of. Figure 5. The SEDFIT solution is nonzero between 42 and 90. The 
solution to our proposed scheme has peaks at 45, 55, 66, and 89, and vanishes 
in between. To verify that these peaks reflect the antibody solute (i.e., the peaks 
are real), we compute the maximum entropy solution for 1000 measurements on 
an equidistant grid, cf. Figure 6. This SEDFIT solution at this higher resolution 
has peaks around 45, 55, 66, and 89. Thanks to the sparsity promoting feature 
of our scheme, we "see" these peaks already with the much broader resolution of 
only 100 measurements. We observe that starting the iteration with the SEDFIT 
solution rather than the zero vector in (7.3) still leads to the same solution which 
indicates an intrinsic stability of the proposed scheme. 

These results suggest that our proposed scheme has great potential when sam- 
ples can be assumed to be highly pure. The method then leads to sharper spikes, 
fewer nonzero entries, and higher resolution. 
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Figure 6. SEDFIT solution (maximum entropy regularized) for 1000 data points. 
There are peaks at 550, 660, and 890 (to compare with 100 data points, x-axes needs 
to be divided by 10 and y-axes needs to be multiplied by 10). Thus the peaks of our 
proposed method at 55, 66, and 89 are real and have the correct amplitude. They 
therefore reflect the antibody solute while using only a tenth of the data. 



8 Conclusion 

We have addressed variational problems with ^^-constraints for g e (0, 1). In case 
that computation time is crucial as it is in any real-time and on-line application, 
there are no sufficiently fast algorithms to solve them. By considering minimiza- 
tion up to a constant factor, we have overcome this limitation. We avoid costly 
iterative schemes and derive closed formulas for such minimizers. This approach 
provides a tool which makes problems for (/ < 1 more feasible than until now. If 
exact solutions are required, those minimizers can initialize iterative schemes to 
speed up their convergence and to find an accurate local minimum. 

We have then modified the Landweber iteration with shrinkage applied at each 
iteration step in [14] by replacing the shrinkage rule with ^?|f^(^'n, an|fn|'^~') to 
cover q & (0, 1) as well. The proposed scheme has been used to solve the ill-posed 
problem of analytic ultracentrifugation. The results have been compared to the 
standard regularization for the analytical ultracentrifugation introduced in [7,41]. 
We have verified that our proposed scheme can provide sparser solutions with 
sharper peaks, higher resolution and smaller residual. Thus, the scheme provides 
a useful add-on to the standard maximum entropy regularization. 

It is known though that iterative schemes of the type presented in Section (7.1) 
converge relatively slowly and thus the computation time of our scheme is orders 
higher than those in [7,41]. To present a competitive approach that can be incor- 
porated into online applications such as the software -package SEDFIT/SEDPHAT 
[41], the method still needs a major tune up to derive a faster convergence, cf. [15] 
for possible directions. 

For further theoretical foundation, it remains to find general conditions on L 
and on the bi-frame such that F*L*LF is bounded on and to compute the 
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difference between q]^'^ and the exact minimizer of (4.1). It also remains to pre- 
cisely determine the arising constants. For the analytical ultracentrifugation, the 
brute force discretization by means of sampling must still be replaced with a proper 
discretization scheme involving suitable ansatz functions and smoothness spaces. 
It also remains to verify that the Landweber iteration with g-dependent shrinkage 
converges towards a minimizer up to a certain error. We plan to address these 
topics in a forthcoming paper. 
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